The Economic Evaluation of Alternatives (eeoa): Rethinking the Application of Cost-effectiveness Analysis, Multi-criteria Decision-making (mcdm) and the Analysis of Alternatives (aoa) in Defense Procurement

Our primary goal is to improve public investment decisions by providing defense analysts and acquisition officials a comprehensive set of approaches to structure an “Economic Evaluation of Alternatives” (EEoA). This study identifies a significant weakness in the Multicriteria Decision-making (MCDM) approach that currently underpins many contemporary AoAs. While MCDM techniques, and therefore most AoAs, correctly focus on lifecycle costs and operational effectiveness of alternatives, “Affordability” is often only implicitly addressed in the final stages of the analysis. In contrast, the adoption of EEoA encourages decision-makers to include affordability explicitly and up-front in the AoA. This requires working with vendors to build alternatives based on different funding (budget/affordability) scenarios. The key difference between the traditional MCDM approach to AoAs and the EEoA approach is that instead of modeling alternatives from competing vendors as points in cost-effectiveness space, EEoA models alternatives as functions of optimistic, pessimistic, and most likely funding (budget) scenarios. The Decision Map offered to practitioners to structure EEoAs provides a unique opportunity to achieve a significant defense acquisition reform—to coordinate the requirements generation system (JCIDS), Defense Acquisition System (DAS), and PPBE process, to lower the costs of defense investments, and improve performance and schedules. Introduction to the Problem: Making the Case for “Affordability” Our nation’s security, billions of taxpayer dollars, and the survival of our soldiers can all hinge on an Analysis of Alternatives (AoA). Routinely conducted by the US Department of Defense (DoD), the AoA is a key component of the defense acquisition process. Investment 1 This study often uses the term “Analysis of Alternatives” (AoA) in its broad, generic sense. Although focused on defense acquisition, the results of the study apply to any public-sector procurement. It should be clear in context when the term AoAs references major defense acquisition programs (MDAPs) as opposed to the acquisition of major automated information systems (MAISs). = = ==================aÉÑÉåëÉ=^Åèìáëáíáçå=áå=qê~åëáíáçå======== 7 = = decisions supported by AoAs help shape future forces, influence defense spending, and occasionally transform the defense industry. This study points to a significant weakness in the Multiple-criteria Decision-making (MCDM) approach that underpins many contemporary AoAs. The weakness is that while MCDM techniques, and therefore most AoAs, correctly focus on lifecycle costs and the operational effectiveness of individual alternatives, “Affordability” is an after-thought, often only implicitly addressed through a weight assigned to costs. In contrast, the approach recommended in this study encourages analysts and decisionmakers to include affordability explicitly in the AoA. This requires working with vendors to build alternatives based on different funding (budget/affordability) scenarios. Supported by a static, deterministic, multi-stage, constrained, optimization micro-economic production (procurement auction) model described in Section 3 (with the math relegated to the Mathematical Appendix available upon request), this “Economic Evaluation of Alternatives” (EEoA) explicitly addresses affordability up-front. The key difference between the MCDM approach to AoAs and the EEoA approach is that, instead of modeling decision alternatives from competing vendors as points in cost-effectiveness space, EEoA models alternatives as functions of optimistic, pessimistic, and most likely funding (resource/budget) scenarios. Given the current financial crisis and future public-spending challenges, affordability is a growing concern. As a consequence, it is imperative that the DoD gets the best value for every dollar it invests in major defense acquisition programs (MDAPs) or major automated information systems (MAISs). A brief review of the DoD’s high-level, fiscally constrained budget development and acquisition systems highlights the key role that affordability needs to play up-front in any AoA. The Planning, Programming, Budgeting and Execution (PPBE) process is the principal decision support system used by the DoD to provide the best possible mix of forces, equipment, and support within fiscal constraints. Two other major decision support systems complement the PPBE process: a requirements generation system called the Joint Capabilities Integration and Development System (JCIDS) and the Defense Acquisition System (DAS). Based on strategic-level guidance (the National Security Strategy, National Military Strategy, Quadrennial Defense Review, Strategic Planning Guidance, etc.), the requirements generation system reviews existing and proposed capabilities and identifies critical capability gaps. To fill those capability gaps, senior leadership examines the full range of “doctrine, organization, training, materiel, leadership and education, personnel and facilities” (DOTMLPF) (CJCS, 2007, p. A-1; USD (AT&L), 2008, p. 14). Whenever a “materiel” solution is recommended, prospective military investments are identified that serve as the basis for AoAs that underpin the development of new acquisition programs in the Defense Acquisition System (DAS). The DAS provides principles and policies that govern major defense acquisition decisions and milestones. To ensure transparency and accountability, and to promote efficiency and effectiveness, various instructions (e.g., FAR, DFARS, DoD Directive 5000.01, DoD Instruction 5000.02, etc.) specify statutory and regulatory reports (e.g., AoAs) and other information requirements for each milestone and decision point. The primary purpose of PPBE is to make hard choices among alternative military investments necessary for national security within fiscal constraints. As we identify alternative materiel investments that can fill current capability gaps, the requirements generation process (JCIDS) naturally fits into the Planning phase of PPBE.